Papers with model generalisation
A (More) Realistic Evaluation Setup for Generalisation of Community Models on Malicious Content Detection (2024.findings-naacl)
Copied to clipboard
| Challenge: | despite the performance of community models for malicious content detection, misinformation and hate speech continue to propagate on social media networks. |
| Approach: | They propose a new evaluation setup for community models for malicious content detection based on a few-shot subgraph sampling approach to test generalisation of models using local explorations of a larger graph. |
| Outcome: | The proposed evaluation setup outperforms existing models on real-world graphs on a training graph. |
Set-Aligning Framework for Auto-Regressive Event Temporal Graph Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for constructing event temporal graphs have been suboptimal . authors propose a set-aligning framework for the effective utilisation of Large Language Models . |
| Approach: | They propose a set-aligning framework for the effective utilisation of Large Language Models to alleviate text generation loss penalties. |
| Outcome: | The proposed framework surpasses existing baselines for event temporal graph generation. |
Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | a new approach to generate adversarial data is needed to improve question answering models . crowdworkers can fool a model only 8.8% of the time, compared to 17.6% for a trained model without synthetic data. |
| Approach: | They develop a pipeline that generates questions and then filters or labels them to improve quality. |
| Outcome: | The proposed approach improves state-of-the-art on a human-written adversarial dataset by 3.7F1 and improves model generalisation on nine of the twelve MRQA datasets. |